DOCUMENT RESUME 



ED 421 142 



IR 018 862 



AUTHOR 

TITLE 

PUB DATE 
NOTE 



PUB TYPE 
EDRS PRICE 
DESCRIPTORS 



Stanbrough, Mark; Stinson, Bill 

Developing a Customized Teaching Assessment Software 
Instrument . 

1998-00-00 

5p . ; In: "SITE 98: Society for Information Technology & . 

Teacher Education International Conference (9th, Washington, 
DC, March 10-14, 1998). Proceedings”; see IR 018 794. 

Figures are illegible. 

Reports - Descriptive (141) -- Speeches/Meeting Papers (150) 

MF01/PC01 Plus Postage. 

^Computer Software Development; ^Computer Uses in Education; 
Evaluation Methods; Instructional Effectiveness; 
Instructional Improvement; Screen Design (Computers); 
^Teacher Evaluation; Teaching Methods; Time on Task 



ABSTRACT 



The goal of this project was to develop customized teaching 
analysis software that would accurately measure recorded teaching behaviors 
and communicate useful results quickly to the observed teacher with the goal 
of improving teacher performance. A computer software program, "The 
Evaluator, " was developed that uses a Windows interface programmed in Visual 
Basic language. It was designed to effectively measure time on task behaviors 
as well as frequency of teacher related behaviors. "The Evaluator" allows 
collection of data for any observable behavior that can be timed, counted, or 
commented on. The opening format has four sections (evaluation forms, online 
evaluation, evaluation data, reports) , each containing a help portion to 
assist in performing the functions of that section. The program is designed 
to print reports, charts, and graphs for immediate feedback. Evaluators using 
this program are able to use their time more efficiently, primarily after the 
evaluation is complete, for counseling the individuals. (AEF) 
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T eachers learn a great deal through trial and error. Yet, as we continue to teach, it becomes obvious that planning, 
organization, student characteristics, progression of skills, social dynamics and other variables are prominent 
elements in our teaching performance. We must not only be visionary but also analytical in what creates an effective 
learning environment. 



Teachers need to subject themselves not only to 
periodic self-evaluation, but also to the scrutiny of peers, 
superiors, and to their students as well. Subjective ratings, 
anecdotal records, and comments from others are an 
important part of evaluating teaching effectiveness. 
However, many of these methods may lack reliability and 
may not be easily transformed into solutions for a problem 
area. Systematic evaluations are objective. The objective 
data created from observation of frequency of events (such 
as positive feedback or using student names) and duration 
of activity (such as time on task or opportunities for correct 
skill practice) contributes a great deal of information about 
what is actually going on. 

Rationale For Developing Customized 
Software 

There are a variety of pencil and paper assessment 
instruments available for analyzing and evaluating teaching 
performance. The instructors wanted to share with the 
students a variety of educational technologies. Therefore, 
a search for software which facilitates collecting, analyz- 
ing, and reporting of observable teaching behaviors was 
instigated. The advantages of such a computer software 
program that utilizes objective criteria such as time and 
frequency enables a person with minimal training to 
conduct an evaluation of someone’s teaching performance. 
Computerized screen presentations and graphical printouts 
of a teaching performance would create an immediate and 
informative awareness of one’s teaching behaviors. 
Simultaneous situations recordings such as time on task 
and frequency of behaviors could be tracked and tran- 
scribed more easily through software technology than by a 
manual paper-pencil and stop watch operation. 

The current software packages available did not address 
all of our needs. The decision was made to develop our 
software in a Windows environment. The Windows 
interface would make the computer program more user 



friendly than MS-DOS based software available. Our goal 
was to develop customized teaching analysis software that 
would accurately measure any recorded various teaching 
behaviors and communicate useful results quickly to the 
observed teacher with the goal of improving teacher 
performance. 

Development of The Evaluator 

Our objective was accomplished by developing a PC 
computer software program, The Evaluator (Henry and 
Hubbard, 1997), that uses a Windows interface pro- 
grammed in Visual Basic language. It was designed to 
effectively measure time on task behaviors as well as 
frequency of teacher related behaviors. The Evaluator 
allows collection of data for any observable behavior that 
can be timed, counted or commented on. Specific 
behaviors to be measured can be determined by the 
observer prior to actual observation. 

Figure 1 shows the opening format. The window has 
four sections. Each section contains a help portion to assist 
in performing the functions of that section. 
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Figure 1 . The Evaluator Main Screen. 
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Evaluation forms. Creates, modifies, and prints 
evaluation forms. 

Online Evaluation. Evaluation data is entered while 
observing using a developed online copy of an evaluation 
form. 

Evaluation Data. Evaluation data is entered from a 
printed hard copy of an evaluation form or to modify 
existing data. 

Reports. View and print reports generated from 
evaluation data in statistical and graphical representation. 

Figure 2 shows a form that has been designed in 
advance to record the specific observations of a teacher’s 
lesson performance. 




Figure 2. The Evaluator Evaluation Form. 



This software program can be used with a PC or laptop 
computer input directly in the classroom/gymnasium or in 
conjunction with paper/pencil recording with later transfer 
to the software program using an online form such as 
figure 2. The program is designed to print reports, charts 
and graphs for immediate feedback. Figure 3 is an 
example of a completed evaluation of a teacher’s perfor- 
mance. This report was instantly processed and displayed 
for immediate feedback to that teacher. 
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Figure 3. The Evaluator Online Completed 
Evaluation Form. 



In addition, figure 4 is a graphical representation of 
observations recorded in that lesson. Graphics can help a 
teacher get a better “picture” of how the lesson progressed. 
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Figure 4. The Evaluator Graphic Report. 



The advantages of a computer software program that 
utilizes objective criteria such as time and frequency are 
obvious. An individual possessing minimal training can 
conduct an evaluation of another person’s performance. 

The computerized screen presentation and/or printout with 
a graphical display of a performance are immediate and 
informative devices which give someone immediate 
awareness of their teaching behaviors. Simultaneous 
situations (such as time on task, definitive portions of a 
lesson - introduction, review, etc.) as well as the time span 
of those situations can be tracked more easily through the 
software’s technology rather than manually through a 
person’s paper/pencil and stop watch procedures. 

Final Thoughts 

Evaluators using The Evaluator are able to use their 
time more efficiently, primarily after the evaluation is 
complete for counseling the individuals. The work is 
virtually done after the evaluation. All of the reports are 
generated automatically and immediately by The Evalua- 
tor. The reports can be graphically demonstrated to allow 
better understanding. Immediate feedback is the major 
contributor to behavior change. The immediate feedback 
of The Evaluator's reports increases the likelihood that an 
evaluation will result in positive behavioral changes. The 
Evaluator can identify specific areas for improvement of 
individuals being evaluated. The objective accuracy of 
The Evaluator means more confidence in the results and 
the more likely they will be accepted. As the performance 
of a person being evaluated improves through the use of 
The Evaluator, the performance levels of others will also 
improve. The Evaluator can also be used to measure time 
on task and other learning behaviors of teachers, fhe same 
benefits of evaluation, immediate feedback and positive 
behavior change that applies to teachers can also be 
applied to students. A demonstration version and infor- 
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mation on the computer software program The Evaluator , 
may be obtained at web site: http://www2.southwind.net/ 
-guerilla or by contacting Guerrilla Software, 6826 East 
Odessa Ct., Wichita, KS 67226. 
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